Because of the diversity of digital data processing equipment types and data formats employed by such equipments, in order to exchange digital information over a communications network between different signal processing devices, it is necessary specify a given protocol through which the data being processed or supplied by a given device is assembled into a preestablished format for transmission from a first site over a communication link to a second site, where the data is recovered and output to a destination device.
FIG. 1 diagrammatically illustrates a typical example of a digital communications system (e.g. a digital telecommunications system capable of handling voice, video and data signals) through which digital data communications are conducted between respective network sites, identified as a west transceiver site 11 and an east transceiver site 13, over a digital communications network 15. West transceiver site 11 includes a digital communications transceiver 20, such as that diagrammatically illustrated in FIG. 2, which has local user-associated receive and transmit ports 21 and 22, respectively coupled via an incoming link 24 from the west user's data terminal equipment 27, and an outgoing link 25 coupled to the west user's data terminal equipment. The digital communications transceiver 20 of the west transceiver site 11 also has network-associated transmit and receive ports 31 and 32, respectively coupled via an outgoing or transmit link 34 to the network 15, and an incoming or receive link 35 from the network.
Similarly, east transceiver site 13 includes its own digital communications transceiver 40, again as shown in FIG. 2, having local user-associated receive and transmit ports 41 and 42, respectively coupled via an incoming link 44 from the east user's data terminal equipment 47 and an outgoing link 45 to the east user's data terminal equipment 47. The digital communications transceiver 40 of the east transceiver site 13 also has network-associated transmit and receiver ports 51 and 52, respectively coupled via an outgoing or transmit link 54 to the network 15, and an incoming or receive link 55 from the network.
As diagrammatically shown in FIG. 2, a respective one of the transceivers 20 and 40 employed at the west and east sites of the network of FIG. 1 includes a digital data transceiver unit (such as a Model 68302 bidirectional digital communications device, manufactured by Motorola Inc.), which contains first and second transmit/receive pairs of serial communications controller (SCC) units 60 and 70 and an attendant microprocessor 80 (including associated memory/buffer storage for incoming and outgoing data frames). User-associated SCC 60 includes a pair of transmit and receive serial communications controllers (SCCs) 61 and 62 coupled to respective transmit and receive ports 63 and 64 on the user side of the transceiver. Similarly, network-associated SCC 70 includes a pair of serial communications controllers 71 and 72 coupled to transmit and receive ports 73 and 74 on the network side of the transceiver.
Digital data sourced from a user's data terminal equipment to be transmitted over the network 15 at a prescribed baud rate (e.g. 56 KBaud) is packetized by the transceiver into limited frame size (e.g. 256 bytes/frame), each data frame being formatted in accordance with a prescribed digital data encapsulation protocol (e.g. High level Data Link Control or HDLC) and an optional compression mechanism (e.g. conventional STAC compression).
Because the data to be transmitted, such as a multipage text file supplied from a user's DTE, is generally comprised of a variable length serial string of digital bytes, which is applied to the local receive port 64, it is initially written into a relatively large capacity (e.g. 10 KBytes) user buffer, the capacity of which is preferably sized to accommodate the largest anticipated user data frame. The transmission control routine employed by the transceiver's microcontroller 80 is operative to cause successive portions of the buffered incoming data stream to be assembled into limited size data frames (256 bytes/frame) for network transmission in accordance with the requirements of the given network, so that serial packetized data is transmitted from transmit port 73 out over the network link 15. Although, for purposes of the present description, Ethernet protocol is referenced as a typical scheme for digital communications requiring transport over the network, it should be observed that the invention to be described below is not limited to use with this or any other protocol.
As diagrammatically illustrated in FIG. 3, a typical HDLC frame 101 of Ethernet user data may include an overall data portion 100, which is made up of an address field 103, containing respective source and destination address sub-fields 105 and 107, an Ethernet type field 111, a variable length (multi-byte) data field 113, and a frame check sequence (FCS) field 115 (comprised, for example, of a pair of frame check sequence bytes 115-1 and 115-2). Leading (beginning) and trailing (terminating) frame boundaries are respectively demarcated by a start-of-frame flag (F) byte 102 and an end-of-frame flag (F) byte 116.
FIG. 4 shows a simplified version of the HDLC data frame of FIG. 3, in which the data portion 100 of a frame is diagrammatically illustrated as comprising a sequence 140 of respective data bytes A, B, C, . . . , and terminating in two frame check sequence bytes F1 and F2.
In order for transceiver equipment at a remote end of the network to properly assemble successive frames of data and direct the data to a destination device, it is necessary to include, as part of a network frame, control information employed by the transceiver for this purpose. As diagrammatically illustrated in FIG. 5, when point-to-point protocol (PPP) is employed to encapsulate the data 140 into successive data frames, a series of header bytes (including one or more protocol identifier (PID) and parameter bytes), shown at 121, is appended to the data portion 110 to generate a message segment 124. The header 121 is located within the message segment, such that the bytes containing the protocol identifier (PID) 126 are also the first bytes in the segment 124, so that the form of any segment can be determined at the receiver site simply by examining the first bytes.
Since the encoding mechanism is recursive, a segment group may be processed and the result treated as through it is simply data. Compressed as shown in FIG. 6, compressed data 127 is a processed version of segment 124 of FIG. 5, so that the addition of the header 130, containing a PID portion 128 that specifies the type of signal processing that has been performed, results in a `nested` header.
For network usage efficiency, the Ethernet segment 124 of FIG. 5 is preferably compressed into a homogeneous compressed sequence, shown at 131 in FIG. 6. Specifically, after appending the PPP Ethernet header 121 to the data portion 110 to create Ethernet segment 124 of FIG. 5, the Ethernet segment 124, in turn, is processed by means of a compression algorithm into a plurality of `compressed` data bytes, shown at 127 in FIG. 6, and a PPP compression header 130 (containing compression PID 128) and an associated data frame sequence number byte 129 are appended to the compressed data bytes 127 to produce a nested compressed segment 131, which is transmitted over the network.
The encoding and compression technique represented by the data frame formats of FIGS. 3-6 enjoys a number of apparent advantages. The first is its simplicity, in that, although there may be many different possible PIDs, it is only necessary to consider them one at a time. A second advantage is the fact that the technique is reasonably flexible, since the receiver at the remote site does not need to know how to decode a received message prior to actual receipt of the message, as each segment contains a PID detailing the decoding procedure. Thirdly, since the PIDs are mutually independent, the addition of new PIDs allows the procedure to be expanded without changing existing PIDs.
On the other hand, a fundamental problem with the above described technique is its poor network bit efficiency. More particularly, although data compression is useful for reducing the number of network bits that must be transmitted, if the Ethernet frame 101 is relatively small, then the apparent gain from compression is lost by the overhead imparted by the segment headers 130 and 121 specifying the decoding procedure. Namely, the problem of efficiency is a direct result of the advantages of the scheme.
One proposal to circumvent this inefficiency problem is simply to ignore it. This actually works well on data having large message segments, since the overhead associated with the headers is fixed, so that the overhead is only a small percentage of the total number of bits transmitted over the network. A second approach is to reduce the length of the header by using a carefully chosen encoding mechanism. One example of this technique would be to pack all parameter flag bits into a single byte.
Another solution involves shortening the header by taking something out. Unfortunately, this generally reduces one of the above advantages. For example, decreasing the number of PID bits also decreases expandability, due to the fact that the number of available PIDs is also reduced. Another alternative strategy is to compress some of the header bytes together with the data being formatted. In fact, the above-described compressed Ethernet example performs this operation, with the nested header 121 containing the source Ethernet packet being located within the compressed part of segment 131. However, this scheme has two shortcomings.
First, the header 121 itself is often effectively incompressible, since it is already carefully encoded. Compressing an effectively non-compressible code generally results in a net loss of bit efficiency. Secondly, since the header 121 is generally a statistically different type of data field than the data 110 itself, using two types of data may make the compression operation produce more bits for the processed data. A further solution is to combine PIDs--for example, by creating a further opcode to indicate both Ethernet packetizing and compression. While such a scheme will work, it reduces flexibility and expandability.